1
Deriving Value from Patient Flow Analytics
Session 200, February 14, 2019
Justin Boyle & Sankalp Khanna
CSIRO Australian e-Health Research Centre
www.csiro.us
2
Justin Boyle, BEng(Hons1) PhD, Principal Research Scientist
Has no real or apparent conflicts of interest to report.
Sankalp Khanna, BEng,MInfTech, PhD, Senior Research Scientist
Has no real or apparent conflicts of interest to report.
Conflict of Interest
3
Learning Objectives
Introduction Where we are from
The Australian Health System Why we do what we do
Patient Flow Analytics @ CSIRO Our approach
Case Studies
Agenda
4
Identify strategies to ensure effective impact from developed
health analytics solutions
Form effective partnerships between problem solvers and problem
owners to ensure success
Apply appropriate statistical rigor in planning and executing data
science projects
Identify strategies to decongest the health system
Learning Objectives
5
Australia’s National Science Agency
Over 5000 Research Scientists
58 Sites globally, Research activities in 80 Countries
$1 Billion Annual budget
Top 1% Of Global research agencies
Hosts Boeing’s largest R&D facility outside of the US
Run NASAs spacecraft tracking facilities in Australia
Invented WiFi, used in five billion devices globally.
CSIRO: Commonwealth Scientific
and Industrial Research Organisation
6
CSIRO’s Digital Health Research Program
BIOMEDICAL
INFORMATICS
Biostatistics, imaging and
genomics based -clinical
workflows
How: Leveraging operational &
clinical data through analytics,
modelling, decision support &
automation
HEALTH
INFORMATICS
Improving health system
performance & productivity from
electronic health data
How: Meaningful data
interoperability and analysis for
decision support, analytics,
modelling and reporting
HEALTH
SERVICES
Improving access to services &
management of chronic
diseases
How: Service delivery models
utilising telehealth, mobile
health & remote monitoring
7
The Australian Health System
Over the Past DecadeOver the Past Decade
Health
Spending
Population
Growth
50%50% 17%17%
Reference: Australia's health 2018, https://www.aihw.gov.au/reports/australias-health/australias-health-2018/
8
9
Overcrowding in hospitals: an
international crisis
- Increased wait times. - Increased walkouts.
- Increased medical errors. - Ambulance diversion.
- Increased length of stay. - Patient safety at risk.
- Increased medical negligence claims - Unnecessary deaths.
ED
admissions
Elective
surgery
Reference: American College of Emergency Physicians, Emergency Department Crowding: High-Impact Solutions, 2008
10
Do we have a problem with Patient
Flow ?
OVERCROWDED HOSPITALS
AGEING POPULATIONS
EXPLODING HEALTH BUDGETS
AMBULANCE RAMPING
LONG WAITING TIMES
STAFFING SHORTAGES
Image Source: Fairfax Media
11
Fixing Patient Flow - Why does it matter?
Reference: Boyle J, Zeitz K, Hoffman R, Khanna S, Beltrame J. Probability of severe adverse events as a function of hospital occupancy. IEEE J Biomed Health Inform. 2014 Jan;18(1):15-20
0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 1.15
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Occupancy
Probability of SAC1/SAC2 Incident
No event/day
1 event/day
2 events/day
3 events/day
12
Patient Flow Analytics @ CSIRO
Improving public hospital performance
through efficiency improvements
Improving public hospital performance
through efficiency improvements
Creating an evidence base to support
policy and decision making
Creating an evidence base to support
policy and decision making
13
Better Capacity Management
85% occupancy delivers optimum patient flow
Higher levels result in increased patient risk and regular overcrowding
Is 85% a one size fits all ?
How do I manage my hospital capacity ?
How unsafe is my crowded hospital ?
Early Discharge
Early discharge should help ease crowding.
Where is the evidence ?
How much benefit could it potentially deliver ?
Hospital Crowding: The Magic Fix
14
Q > How full can I run my Hospital
at before Capacity Crisis ?
Case Study 1 : Capacity Management
ED, Admission & Discharge vs occupancy
3 key choke points - performance declines:
A - Admission/discharge surge
B - ED overwhelmed
C - Admissions overwhelmed
Overcrowding affects :
Access Block
ED Length of Stay (Inpatients)
Inpatient Length of Stay
Inpatient Admissions from ED
ED Length of Stay (not admitted)
15
Q > Do we see this trend across hospitals of all sizes ?
Case Study 1 : Capacity Management
0
5
10
15
20
25
30
35
40
45
50
0
10
20
30
40
50
60
70
80
90
71%
80%
89%
98%
107%
OCCUPANCY
Inpatient Admissions (patients/hr) (Y1 axis)
Inpatient Discharges (patients/hr) (Y1 axis)
ED Presentations (patients/hr) (Y1 axis)
ED Discharges (patients/hr) (Y1 axis)
Inpatient Admissions from ED (patients/hr) (Y1 axis)
Inpatient Length of Stay (days) (Y2 axis)
ED Length of Stay (inpatients) (hours) (Y2 axis)
ED Length of Stay (others) (hours) (Y2 axis)
ED Access Block Cases (inpatients) (patients/hr) (Y2 axis)
0
5
10
15
20
25
30
35
40
45
0
10
20
30
40
50
60
70
80
90
75%
80%
85%
90%
95%
100%
105%
110%
115%
OCCUPANCY
GROUP 3
A
B
C
300 >= Beds
0
5
10
15
20
25
30
35
40
45
0
10
20
30
40
50
60
70
80
90
75%
80%
85%
90%
95%
100%
105%
110%
OCCUPANCY
GROUP 2
A
B
C
900 >= Beds > 300
0
5
10
15
20
25
30
35
40
45
0
10
20
30
40
50
60
70
80
90
70%
75%
80%
85%
90%
95%
100%
OCCUPANCY
GROUP 1
A
B
C
Beds > 900
YES … but at different levels
Group 1 (Large hospitals) :
Choke Point A : 86%
Choke Point B : 90%
Choke Point C : 94%
Group 2 (Mid-size hospitals) :
Choke Point A : 90%
Choke Point B : 96%
Choke Point C : 101%
Group 3 (Small hospitals) :
Choke Point A : 98%
Choke Point B : 102%
Choke Point C : 106%
16
Case Study 2 : Discharge Timing
Q > Does discharge peak timing affect ED LOS and Access Block
5 hours
Admissions
Discharges
d1’
5 hours
Discharges
d2’
Category 1 Category 2 Category 4 Category 5
Category 3
Hour of Day
Number of Patients
Define
discharge
peak timing
Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19
th
Australian National Health Informatics Conference (HIC 2011), 2011, 82-88
17
Case Study 2 : Discharge Timing
Q > Does discharge peak timing affect ED LOS and Access Block
Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19
th
Australian National Health Informatics Conference (HIC 2011), 2011, 82-88
0
50
100
150
200
250
75%
80%
85%
90%
95%
100%
105%
110%
115%
1
2
3
4
5
Access Block Cases per day
Occupancy (%)
Category
23 Hospitals
Mean Occupancy (Y1 Axis)
Mean PeakOccupancy (Y1 Axis)
Mean AB Cases (Y2 Axis)
6.0
6.5
7.0
7.5
8.0
8.5
9.0
9.5
10.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
1
2
3
4
5
Length of Stay (hours)
Length of Stay (days)
Category
23 Hospitals
Mean LOS (days)
Mean EDLOS (hours)(Y2)
Cat 5 vs Cat 1
13% Higher Peak
Occupancy
60 cases/day higher
Access Block
0.7 hours higher
Mean ED LOS
18
Case Study 2 : Discharge Timing
0
50
100
150
200
250
300
350
400
450
55
60
65
70
75
80
85
90
95
100
105
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Discharges/hour
Occupancy (%)
Time of Day (hour)
2 Hours Early
1 Hour Early
Actual
1 Hour Late
2 Hours Late
2 Hour Early Discharge (all 23 Hospitals) :
Average Occupancy reduced from 93.7% to
91.6%.
Maximum Occupancy reduced from 110.8%
to 106.1%.
Time spent above 95% occupancy reduced
from 34.7% to 21.5%.
Q > Can we quantify the impact of Early Discharge ?
Q > What happens if overcrowding delays Discharge ?
Reference: Khanna et al. Unravelling relationships: Hospital occupancy levels, discharge timing and emergency department access block. Emerg Med Australas. 2012 Oct;24(5):510-7
19
Case Study 2 : Discharge Timing
Q > How does this affect my KPIs ?
Q > How can I operationalise this ?
Scenario
Description
1
50% of patients to be discharged by 10am, 80% by 12pm and 100% by
2pm.
2
35% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.
3
50% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.
4
80% of patients to be discharged by 11am.
5
40% of patients to be discharged by 10am, 70% by 2pm, 90% by 5pm and
100% by 10pm.
6
Select the same patients as for
Scenario 5 but change only the emergency
discharge times, leaving elective patient discharge times unchanged.
7
Select the same patients as for Scenario 3 but change only the emergency
discharge times, leaving elective patient discharge times unchanged.
Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70
20
Case Study 2 : Discharge Timing
Q > How does this affect my KPIs ?
Q > How can I operationalise this ?
Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70
% change as compared to baseline
Scenario
Scenario Description
NEAT
Performance
Ave Bed
Occupancy
Ave Inpatient
LOS
Ave wait for
Inpatient Bed - ED
Ave wait for
Inpatient Bed - All
1
50%
by 10am, 80% by
12pm,
100
% by 2pm.
+16.1% -1.5% -1.7% -25.5% -24.2%
2
35%
by 11am, 70% by 2pm,
100
% by 5pm.
+5.7% -0.2% -0.3% -6% -5.7%
3
50
% by 11am, 70% by 2pm ,
100
% by 5pm.
+9.4% -0.5% -0.5% -11.8% -10.5%
4
80%
by 11am. +16.2% -1.5% -1.6% -24.9% -23.5%
5
40%
by 10am, 70% by 2pm,
90% by
5pm, 100% by 10pm.
+7.3% -0.3% -0.4% -8.6% -7.7%
6
Same as Scenario 5
but ED only
+6.9% -0.2% -0.3% -7.3% -6.4%
7
Same as Scenario 1
but ED only
+15.7% -1.2% -1.3% -22.7% -20.5%
21
How Many of What Beds?
Access Target:
ED LOS<4hrs
22
Case Study 3 : Predicting Demand
Forms a regular component
of daily bed management
across major QLD public
hospitals
Licensed in Australia and
overseas
Estimated to deliver $23
million (direct), and $250m
(indirect) in productivity gains
per annum
Several awards related to
efficiency and effectiveness
References:
Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.
Emerg Med J. 2012 May;29(5):358-65.
Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,
Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331
Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient
admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32
23
Case Study 3 : Predicting Demand
Forms a regular component
of daily bed management
across major QLD public
hospitals
Licensed in Australia and
overseas
Estimated to deliver $23
million (direct), and $250m
(indirect) in productivity gains
per annum
Several awards related to
efficiency and effectiveness
References:
Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.
Emerg Med J. 2012 May;29(5):358-65.
Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,
Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331
Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient
admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32
24
Case Study 3 : Making it Work
WHO?
Bed manager, after hours co-
ordinator, hospital executive,
hospital executive on-call, Director
ED, Director Medicine, Director
Surgery, decision support services
WHY?
Inform managers (nursing and
medical) re expected admissions
and discharges so they have
information to work proactively
HOW?
PAPT Software and PAPT
Procedure Manual: guide to assist
communication processes with in-
built ‘triggers’ to inform decision
making (planning and functioning)
WHEN?
Daily and Weekly
Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on
Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76
25
Case Study 3 : Making it Work
WHO?
Bed manager, after hours co-
ordinator, hospital executive,
hospital executive on-call, Director
ED, Director Medicine, Director
Surgery, decision support services
WHY?
Inform managers (nursing and
medical) re expected admissions
and discharges so they have
information to work proactively
HOW?
PAPT Software and PAPT
Procedure Manual: guide to assist
communication processes with in-
built ‘triggers’ to inform decision
making (planning and functioning)
WHEN?
Daily and Weekly
Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on
Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76
26
Case Study 4 : Evidence Driven KPIs
The National Emergency Access Target (NEAT)
“By 31 Dec 2015, 90% of all patients will physically leave the
Emergency Department (ED) within 4 hours
Australian Institute of Health and Welfare Canberra, Australian
hospital statistics 201213, Emergency department care,
Health Services Series Number 52
64%
66%
72%
77%
66%
67%
64%
57%
64%
66%
72%
77%
66%
67%
64%
57%
First study to deliver evidence driven
targets for public hospital ED patient flow
Directly translated into government policy
Several awards and endorsements
Replicated in several other states, and in
the UK
Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)
and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354
27
Case Study 4 : Evidence Driven KPIs
Methodology :
Focus on Emergency Admissions only
(i.e. eHSMR)
Exclude Palliative Care and Short Stay
Develop several predictive models for
eHSMR calculation
Model relationship between NEAT
Compliance and eHSMR
Check for confounding effect of palliative
care and short stay
No robust evidence regarding a clinically
significant mortality benefit above this threshold
Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)
and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354
28
Case Study 5 : Reducing Readmissions
Project Brief
To develop, implement and evaluate a
web-based risk stratification algorithm
that can be used in-hospital to identify
chronic disease patients with a high risk
of re-hospitalisation.
What are we predicting
Unplanned re-admission within 30
days of discharge from hospital
Unplanned ED re-presentation within
30 days of discharge from hospital
Timeline
Trial - Apr 2018 to Mar 2019
Evaluation - Apr 2019 to Jun 2019
Chronic Disease Patient Admitted
to Hospital
Risk Score generated overnight
Risk score used by care teams for
appropriate interventions and
care/discharge planning
Next morning
29
Case Study 5 : Reducing Readmissions
Patients who, over a 5 year period, as an
Emergency or Admitted Patient :
Attended Logan Hospital, and
Had at least one Chronic Disease visit
(any QLD hospital)
Patient Cohort
Emergency Data (EDIS/FirstNet)
Inpatient Data (QHAPDC/ePADT)
Mortality Data (Death Registry)
Pharmacy dispensing information (eLMS)
Pathology test results (AUSLAB)
Data Used for Modelling & Validation
Patients stays in prev. 180 days
ED presentations in prev. 180 days
Marital status
Age
Indigenous status
SEIFA
Admission source
Admission unit
Care type
Elective status
Planned same day status
Binary flags for routine dialysis
Number of medication records
Binary flags for medication
Binary flags for abnormal pathology
Predictor Variables in Final Models
Logistic regression
Naïve Bayes
Neural Nets
Random Forests
Generalised Boosting
Modelling Techniques
30
Case Study 5 : Making it Work
Employing Intelligible Machine Learning
31
Case Study 5 : Making it Work
Employing Intelligible Machine Learning
32
- Engagement between with clinicians when designing solutions
- Focus on translation early on
- Frameworks, Standards and Governance
- Ensuring statistical rigor
- Choosing the right outcome measures KPIs vs patient outcomes
- Translation is not the clients problem
- Innovative approaches
- Value of understanding the domain and the data
- Empathy with perspectives of the problem owner and the pain points
Recommendations
33
For more information, please contact:
www.csiro.us
Please remember to complete the online session evaluation
Questions
Justin Boyle
Principal Research Scientist
e Justin.Boyle@csiro.au
Sankalp Khanna
Senior Research Scientist
e Sankalp.Khanna@csiro.au
t @SankalpKhanna
l /in/sankalpk